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Abstract 


This paper describes a stand-alone network-based 
video capture and processing peripheral (the Vid- 
board) for a distributed multimedia system centered 
around a gigabit-per-second Asynchronous Transfer 
Mode (ATM) network. The Vidboard captures video 
from an analog NTSC television source and transmits 
it to devices within the system. Devices control the 
Vidboard through a set of ATM protocols. Whereas 
capture boards typically generate video streams having 
fixed frame rate characteristics, the Vidboard is capa- 
ble of decoupling video from the real-time constraints 
of the television world. This allows easier integration 
of video into the software environment of computer sys- 
tems. The Vidboard is based on a front-end frame- 
memory processor architecture that is also capable of 
generating full-motion video streams having a range 
of presentation (picture size, color space) and network 
(traffic, pixel packing) characteristics. A series of ex- 
periments are presented in which video is transmitted 
to a workstation for display. Frame rate performance 
and a remote video source control model are described. 
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1 Introduction 


Asynchronous Transfer Mode [5] (ATM) is a communi- 
cation paradigm that offers support for seamless broad- 
band communication across heterogeneous networking 
environments, from wide-area to the desk-area [8]. ATM 
has many properties which make it well suited to 
the transport of real-time (audio and video) informa- 
tion [1]. In turn, many research groups are designing 
distributed multimedia systems centered around ATM 
networks [12, 6, 4]. 


With the use of ATM, a new class of devices is pos- 
sible in which devices act as shared network resources 
and communicate with each other through ATM-based 
protocols. In terms of multimedia, examples of such de- 
vices are video and audio capture boards, frame stores 
and video servers. 


This paper describes an ATM network-based video 
capture and processing peripheral device called the Vid- 
board. The Vidboard was designed in the context of 
the ViewStation project [12]. The ViewStation sys- 
tem is an all-digital distributed multimedia system cen- 
tered around a gigabit-per-second ATM network. The 
Vidboard was developed as a capture interface for the 
ViewStation environment. Full-motion video is cap- 
tured from an analog television source and transmitted 
to other devices within the system. In terms of research, 
it was developed as a system level prototype for study- 
ing the properties of a network-based video source for a 
distributed ATM environment. These properties relate 
to functionality and ATM control mechanisms. 


Video capture boards have been proposed /designed 
for other research-based distributed multimedia systems 
similar to the ViewStation. In the Pandora system [7], 
a multimedia box containing a video capture board 
is loosely coupled to a workstation. Black and white 
video in a variety of picture sizes is transmitted be- 
tween boxes over a dedicated high-speed network. In 
the Desk Area Network Project [6], an ATM network- 


based capture board with minimal functionality is de- 
scribed. At the University of Washington [4], a network 
capture board which generates video streams consisting 
of NTSC samples has been developed. Workstation- 
based capture boards have been developed for comput- 
ing environments consisting of networked IBM RS6000 
workstations [3, 13]. 


In these systems, video information is hidden from the 
user workstation’s processor and network to varying de- 
grees. In many cases, this hiding occurs by separating 
video from other types of information through the use of 
dedicated hardware, such as dedicated high-speed net- 
works and display mixing cards. When video is mixed, it 
is usually transported in a non-pixel representation (eg. 
compressed) that is not directly usable by the work- 
station processor and is sent to dedicated hardware for 
processing. The rationale for hiding video has been per- 
formance - to avoid saturating workstation and network 
resources. One of the primary goals of the ViewStation 
project is to process video at the application level [12]. 
This software intensive approach will become feasible in 
a local environment given the trends in workstation and 
network performance, and allows for easy migration to 
higher performance workstations as they appear. 


Operating in this software intensive environment 
places temporal demands on the video streams gener- 
ated by video sources. Within this environment, video 
tends to be processed in bursts and processing speed 
depends on workstation resource availability. A mecha- 
nism needs to exist for adapting the frame rate to per- 
mit graceful degradation as system resources become 
scarce. The capture systems described above were de- 
signed to produce video streams having constant frame 
rate characteristics over the course of a video session. 
The Vidboard architecture is novel in that it allows 
the decoupling of video from the real-time/synchronous 
constraints of the television world. Through a closed- 
loop control mechanism, a destination device can dy- 
namically vary the frame rate during the course of a 
video session. The Vidboard architecture also allows 
the generation of full-motion video streams having var- 
ious presentation (picture size, color space, etc.) and 
network transport characteristics. 


The remainder of this paper describes the design and 
use of the Vidboard. Section 2 describes the ViewSta- 
tion system and discusses the major functional objec- 
tives which motivated the design of the board. Section 
3 describes the hardware and software components of 
the Vidboard. Section 4 describes a set of protocols 
for transmitting video from the Vidboard to a worksta- 
tion. Section 5 discusses two systems experiments and 
presents performance results. Section 6 draws a number 
of conclusions from the Vidboard research and describes 
future work. 
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Figure 2: Application-friendly video source model 


2 Background 


2.1 The ViewStation system 


As illustrated in Figure 1, a typical ViewStation system 
interconnects general purpose workstations and special- 
ized video resources, such as Vidboards, image process- 
ing systems, frame stores and video servers. Commu- 
nication within the system is supported by a gigabit- 
per-second ATM network called the VuNet!. The 
VuNet consists of switches interconnected by fiber links. 
Clients connect to the network through a switch port. 
See [12] for a more detailed description. 


2.2 Objectives 


As described previously, the software-intensive nature 
of the ViewStation environment places constraints on 
video sources. An application must have a mechanism 
for adapting the frame rate to system resources. The de- 
sign of the Vidboard was motivated by the application- 
friendly video source model depicted in Figure 2. In this 
model, the application sends the Vidboard control infor- 
mation, which is processed by an intelligent video agent, 
and the Vidboard returns a video stream. Through 


Within the ViewStation system, the ATM protocol is modi- 
fied by adding 3 bytes to the end of the header so that the overall 
length of a cell is 56 bytes, a length that is 4 byte aligned. The 
additional 3 bytes remain unused. 


this closed-loop technique, the workstation regulates the 
characteristics, and in particular the frame rate, of the 
video stream. 


This model led to the following design objectives for 
the Vidboard: 


Temporal decoupling: The ability to process video 
at a rate which is disjoint from that imposed by 
television video. The Vidboard serves as an inter- 
face between the real-time world of television and 
the virtual-time ViewStation computing environ- 
ment. 


Uncompressed video: The Vidboard generates digi- 
tal video in raw form. The reasons for using un- 
compressed video are twofold: network bandwidth 
is not expensive in the ViewStation system and 
video data is handled at the application level. The 
current generation of workstations does not have 
the processing power to handle the high data rates 
of uncompressed video, limiting the frame rate of 
video streams. However, the ViewStation environ- 
ment can be easily ported to higher performance 
workstations as they appear, alleviating this prob- 
lem. This argument does not mean that compres- 
sion does not play a role in the handling of digi- 
tal video. It simply states that compression is not 
appropriate for shipping video from the Vidboard 
to an application running on a workstation. Ar- 
eas where we have found compression to be appro- 
priate are storage and communication across low 
bandwidth networks attached to the ViewStation. 


Distributed control: The ability to receive and exe- 
cute commands from other devices within the sys- 
tem. 


Network transport options: The ability to tailor 
the video stream to the needs of the destination 
device on a network transport level. Two factors 
are involved: the packing of video data into cells 
(transport protocol) and the generation of video 
streams having different traffic characteristics. 


Another set of objectives relates to digital video ap- 
plications: 


Full-motion video: The ability to generate full- 
motion (30 frames/s) video streams. Full-motion 
video streams are important from the point of view 
of applications. 


Presentation options: One level upon which video 
streams can be tailored concerns the presentation 
characteristics of video, which determine how the 


video is displayed. Presentation options include 
picture size and color space. Devices will have dif- 
ferent needs in terms of presentation options for 
application and bandwidth reasons. 


A third set of objectives relates to the network tech- 
nology adopted for the ViewStation system: 


ATM communication: Physical connection to the 
VuNet switch and adherence to the ATM protocol. 


3 The Vidboard 


The Vidboard is based on a front-end frame-memory 
processor architecture as shown in Figure 3. The 
architecture is centered around a Texas Instruments 
TMS320C30 digital signal processor (DSP). In turn, the 
Vidboard is actually a system having both hardware and 
software components. Video is captured by the Front 
End and the resultant pixel information is stored in the 
Frame Memory. The organization of the pixel infor- 
mation within the Frame Memory is controlled by the 
Format Convert circuitry. The pixels are then read by 
the DSP, packed into ATM cells and sent to the net- 
work. The DSP is also responsible for receiving and 
interpreting commands from other devices within the 
ViewStation system. On a research level, the DSP is 
important because, through software, it provides the 
flexibility needed to implement different sets of func- 
tionality. 


In this section, the hardware and software compo- 
nents of the Vidboard are described. The rationale be- 
hind the design of the components is summarized. 


3.1 Hardware architecture 


The Front End executes the actual video capture oper- 
ation on the board. It consists of a digital video chipset 
from Philips. The chipset supports a wide range of fea- 
tures for generating digital video streams with different 
characteristics. The chipset captures one of three NTSC 
television signals? to a resolution of 640x480 pixels. The 
pixels are quantized to 24 bits in either the RGB or YUV 
(luminance-chrominance) color space. 


During transmission, the packing of digital video com- 
ponents into a data stream is an important issue. The 
packing format determines the ease with which a re- 
ceiving device can use the video data. The Vidboard 


?The chipset is also capable of capturing PAL/SECAM video, 
however, the board is designed to capture only NTSC video 
efficiently. 
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Figure 3: Block diagram of the Vidboard 


provides two methods for implementing different pack- 
ing formats. The first method consists of modifying the 
format in the DSP as the pixel stream passes through 
it. This method has the critical disadvantage of pre- 
venting real-time performance since format conversion, 
when executed over an entire video frame, represents 
a computation intensive task. Therefore, the Vidboard 
provides real-time support for different packing formats 
in hardware. The Format Convert circuitry acts as a 
pipeline stage between the Front End and the Frame 
Memory. It takes the three component streams pro- 
duced by the Front End and organizes their writing into 
the 32-bit word Frame Memory. The manner in which 
the components are stored in the memory determines 
how they appear within the DSP word, and ultimately 
influences the packing format of the transmitted video 
stream if the pixel words are transferred directly from 
the Frame Memory to the network. For certain video 
streams, it is most efficient to use a combination of the 
two methods. 


The Format Convert circuitry consists of a crossbar 
switch to route the Front End component streams to 
the proper video RAM (VRAM) chips making up the 
Frame Memory and a state machine made for control. 
Two formats are currently supported: 


e Component/frame format: Each of the three pixel 
components is grouped together across the frame 
and sent in the order of the pixels, one component 
after another. For example, in RGB space, the R 
components for the entire frame are sent, followed 
by the G components and finally the B components. 
This is the preferred format for storage of RGB 
video and, in YUV space, the display of black and 
white video. 


e Component/pixel format: The components that 
belong to each pixel are grouped together and 
transmitted in the order of the pixels. For example; 


for a given pixel, the R component is sent first, fol- 
lowed by the G and B components. Then, the next 
pixel is sent in the same manner and this process 
is repeated over the whole frame’s worth of data. 
This is the preferred format for color display on 
24-bit frame buffers. 


The Frame Memory consists of 3 MBytes of VRAM 
logically organized into two banks, each bank capable 
of storing one video frame. The two memory banks 
are used to buffer the video so that one frame can be 
transmitted while the other is being captured in real- 
time. Digital video data is written into the memory 
through the VRAM serial access memory port on a scan 
line basis and read by the DSP through the conventional 
DRAM port. The DSP has random access to the video 
data which facilitates many of the operations involved 
in video processing such as spatial subsampling. 


The Frame Memory serves as a mechanism for decou- 
pling video from the real-time requirements of televi- 
sion. Since the capture of a video frame into the Frame 
Memory is controlled by the DSP and the Frame Mem- 
ory is double-buffered, the Frame Memory can be used 
as arate adapter between the television rate of the Front 
End digital video stream and the rate at which the DSP 
processes a frame of video. 


The Vidboard is centered around the DSP? which acts 
as a pixel engine, a command dispatcher and the board 
controller. During video transmission, the DSP acts as a 
pixel engine. Digital video data is read from the Frame 
Memory, possibly processed in some way, packaged into 
ATM cells and written to the network. The DSP’s dual 
bus structure allows the video data to flow through the 
processor as it is moved from the memory to the net- 
work. 


3The C30 is a 32-bit floating point, 16.6 MIPs digital signal 
processor that has two independent parallel buses. 


The Vidboard design was fabricated into a printed- 
circuit board in January 1992. The board consists of 
four signal layers and its dimensions are 13.85in by 
9.15in. Cost of a complete board is $1400 in prototype 
quantities. 


3.2 Software framework 


A Vidboard will typically contain a software library of 
the tasks it supports. Devices within the ViewStation 
system make the Vidboard execute tasks by sending 
commands over the network. The DSP runs a sim- 
ple dispatcher for receiving and interpreting these com- 
mands. The dispatcher relies on three mechanisms for 
supporting distributed control: command cells, echo 
back cells and command tables. 


A command cell is an ATM cell generated by the con- 
trolling device and sent to the Vidboard which contains 
information about the task the board is to execute. Ta- 
ble 1 describes the general format of a command cell. 
Examples of command cells which have been created 
for remote debugging are READ and WRITE cells for 
respectively reading and writing the Vidboard address 
space. 


field # of bytes 
ATM cell header 5 
unused portion of header 3 
echo VCI/tport 4 
command code 4 
command parameters AO 


Table 1: General command cell format 


The echo back mechanism is used to prove to a con- 
trolling device that the command it sent was received 
correctly. When a command is received, a copy of the 
command cell (echo back cell) is sent back to the con- 
trolling device, where a check is performed. The echo 
VCI/tport field of the command cell is used to generate 
the echo cell header for routing. 


The dispatcher determines which command to exe- 
cute through the command code field. A number, or 
command code, is associated with each task the Vid- 
board supports. When interpreting a command cell, 
the dispatcher uses the command table to map from 
the code to the corresponding Vidboard routine. 


Besides those for video capture, processing and trans- 
mission, library routines have been developed for remote 
debugging and network diagnostics. Time-critical rou- 
tines, such as those involved in video transmission, are 
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Figure 4: Pseudo-AAL5 transmission frame format 


written in assembler language, while non-time-critical 
routines are written in C. 


4 Video to the workstation pro- 
tocols 


A set of protocols have been developed for transmit- 
ting video from the Vidboard to a workstation. These 
protocols fall into two categories: a command suite for 
controlling the Vidboard and network transport /traffic 
protocols for end-end transport. 


Four types of command cells are used to control the 
Vidboard in a closed-loop control model. They are: 


GRAB: The GRAB cell puts the Vidboard into a free- 
running video capture loop. The next three cells 
control the functioning of the board within this 
loop. 


TRANSMIT: The TRANSMIT cell causes transmis- 
sion of the last frame to be captured. 


FE_INIT: The FE_INIT cell initializes the front-end 
chipset and crossbar chip according to the desired 
presentation options. 


FE_OFF: The FE_OFF cell disables the front-end cir- 
cuitry, terminates the grab loop and returns the 
Vidboard to an idle state. 


In a typical video session, the workstation application 
places the Vidboard into a free-running capture loop 
with the GRAB cell and then selects the desired pre- 
sentation options with the FE_INIT cell. TRANSMIT 
cells are then sent every time a frame is desired. The 
selected presentation options can be changed during the 
course of a session by sending additional FE_INIT cells. 
An FE_OFF cell is sent to end the session. 


To move video data over the VuNet, the Vidboard 
implements a network transport protocol that is simi- 
lar to that specified by the ATM Adaptation Layer 5 
(AALS) standard [2]. As shown in Figure 4, cells are 


grouped into larger packets called transmission frames 
(t-frames)*. Two levels of segmentation occur during 
video transmission. Each video frame is segmented into 
a number of t-frames and each t-frame is further seg- 


mented into a number of ATM cells. 


word # | field description 

0-1 header ATM cell header 

2 echo VCI/tport | for echo cell 

3 command code | grab and transmit 
4 dest VCI/tport | for video traffic 

5) IBD interburst delay 

6 IFD interframe delay 

7 CB cells per burst 

8 LF scan lines per t-frame 
9 SUB spatial subsampling 
10 COLOR color space 

11-13 | not used 


Table 2: TRANSMIT command cell format 


To avoid overwhelming a workstation with network 
data, the Vidboard implements a network traffic pro- 
tocol. Video data is sent in small bursts since unequal 
transmission and reception rates can be tolerated over 
small numbers of cells because the VuNet switch buffers 
incoming traffic. A burst is followed by a delay during 
which the workstation drains the buffer of the burst. 
Video traffic characteristics are defined by four param- 
eters: interburst delay, inter t-frame (interframe) delay, 
cells per burst and scan lines per t-frame. These param- 
eters are specified in the TRANSMIT command cell, as 
shown in Table 2. The parameters are used to shape 
the video traffic to a pattern which can be tolerated by 
the workstation. 


5 Workstation experiments 


The mechanisms described in the previous subsection 
have been used for transmitting video from the Vid- 
board to an Alpha workstation for display in a small- 
scale ViewStation system. The system consists of two 
Vidboards and two Alpha workstations connected to 
one VuNet switch®. The Alphas are interfaced to 
the switch through a simple programmed-I/O interface 
board and are equipped with 8-bit frame buffers. 


On the Alphas, several routines are used to receive 
and display the video stream. The vsdemo program is 


4The word size in the context of the Vidboard is that of the 
DSP, 32 bits. 

5The current implementation of the VuNet switch has four 
ports, and can therefore connect up to four devices. 


an X-windows based video viewer application. It con- 
sists of two windows: a view panel for displaying the 
video and a control panel for selecting the presentation 
characteristics of the video stream. The cell reception 
device driver reads cells from the switch, unpacks them 
according to the pseudo-AAL5 transport protocol and 
stores the contents into memory buffers. 


For color video, since the Alpha frame buffers have 8 
planes, the Vidboard reduces the 24-bit RGB values to 
8 bits. Two techniques for the reduction are explored. 
The first simply consists of quantizing the pixel value 
to 8 bits. The second technique implements a dither 
algorithm to improve the quality of the video picture. 
This technique uses four sets of quantization look-up 
tables which are offset to give the viewer the impression 
of color half-tones. The sets of look-up tables form a 
two-by-two window which is scanned across the video 
frame. 


This section describes two experiments which focus on 
frame rate performance and concludes with a discussion 
of video source control models. 


5.1 Vidboard-workstation frame rates 


Experiments were conducted to determine the video 
frame rate which could be achieved between the Vid- 
board and a Alpha if the latter’s only task is the display 
of video. The frame rates, and corresponding, bit rates 
for various video streams are listed in Table 3. Cer- 
tain of the frame rates are limited by the Alpha, while 
others are limited by the Vidboard. For example, the 
frame rate of the 640x480 black and white video stream 
is limited by the rate at which the Alpha can read from 
the network and display video. The main bottleneck 
to video traffic between the Vidboard and the Alpha 
is the simple programmed-I/O VuNet interface®. As 
the table shows, the maximum video bit rate which can 
be achieved in the context of this system is about 20 
Mbits/s. The Vidboard is the limiting factor in the case 
of the color streams since the quantization and dither 
algorithms are computation intensive. 


5.2 Full-motion experiments 


Full-motion frame rates are not achieved when dis- 
playing video on the Alpha for a variety of reasons. In 
terms of the Alpha, the main issue is the slow speed 
of the I/O bus where the network interface and frame 
buffer are located. In terms of the Vidboard, the main 


6A DMA interface is being developed that will increase the 
network data rate by several factors. 


Video type 


Picture size Black and white | Color (quantization) Color (dither) 
frames/s (Mbits/s) | frames/s (Mbits/s) | frames/s (Mbits/s) 
640x480 9.3 (22.8) 1.8 (44) 1.6 (3.9) 
320x240 28.0 (17.2) 7.7 (4.7) 6.0 (3.7) 
212x160 30.0 (8.2) 16.6 (4.5) 10.0 (2.7) 
160x120 30.0 (4.6) 20.0 (3.0) 15.0 (2.3) 


Table 3: Vidboard to Alpha frame rate performance 


Color space Picture size | Packing format | Frame rate | Video data rate 
(frames /s) (Mbits/s) 
black and white || 640x480 comp/frame 30 74 
320x240 comp/frame 30 18 
24-bit RGB 640x480 comp/frame 20 147 
640x480 comp /pixel 20 147 
320x240 comp /pixel 30 55 


Table 4: Maximal Vidboard frame rates 


issue is the computation intensive nature of the color 
video algorithms. 


A series of experiments were conducted to determine 
the maximum frame rate the Vidboard could generate 
assuming that a ViewStation device existed that could 
receive high-bandwidth video streams. These experi- 
ments consisted of having the Vidboard transmit video 
as fast as possible, with a minimal amount of framing 
information, to an empty VuNet switch port. Frame 
rates were determined by counting the number of cycles 
it took to transmit one frame. The frame rates achieved 
for various combinations of color space, picture size and 
pixel packing format are reported in Table 4. The re- 
sults show that full-motion frame rates are achievable 
in all cases except for full-resolution color’. 


5.3. Video source control models 


As was mentioned previously, a primary objective of 
the Vidboard is to decouple video from the hard time 
constraints present in the television world. In the exper- 
iments presented in the previous subsections, a closed- 
loop control was used to adapt the frame rate of the 
Vidboard to the computational resources of the work- 
station. From a historical perspective, it is of interest 
to analyze the control models developed for requesting 
video. 


In our initial experiments, we implemented a 
connection-based model in which the workstation sent a 


7A faster version (by 20%) of the C30 DSP will be produced 
shortly. This will improve frame rate performance. 


video service command cell and the Vidboard returned 
a constant bit-rate video stream. This model had the 
disadvantage that the workstation was forced to accom- 
modate a constant bit-rate stream no matter what sys- 
tem resources were at its disposal. An example of the 
type of problem which occurred was the partial loss of 
frames at the network interface as the workstation de- 
voted resources to tracking a mouse movement or to a 
disk access. 


Since the connection-based model was not well 
adapted to the ViewStation environment, the closed- 
loop model, which was our initial design goal, was im- 
plemented. The frame rate adaptation mechanism of 
this model was especially apparent when two vsdemo 
programs were executed concurrently, each controlling 
a different Vidboard. The allocation of system resources 
could be seen to lean towards the servicing of one Vid- 
board or the other as different frame rates and picture 
sizes were selected. 


6 Conclusion 


This paper described the Vidboard, an experimental 
network-based video capture board for the ViewSta- 
tion distributed multimedia system. The Vidboard’s 
front-end frame-memory processor architecture allows 
the temporal decoupling of video from the real-time con- 
straints of the television world as well as the generation 
of full-motion video streams having variable presenta- 
tion and network characteristics. 


A number of important implications, concerning the 
properties of a network-based video capture module, 
can be drawn from the Vidboard research. The ability 
to temporally decouple video from its television source 
combined with a closed-loop control model facilitates 
the integration of video into the virtual-time computing 
world. Video service can be adapted to the available 
system resources and degrades gracefully as resources 
become scarce. A second implication is that a capture 
module needs to be able to accomplish tasks related 
to the distributed nature of the environment and not 
only capture/transmit video. Task examples are net- 
work transport, network traffic and distributed control. 


With the trend towards integration of video into the 
digital domain, research groups are developing cameras 
which produce video in a digital format [9, 10] that 
will come to replace television camera/capture board 
systems for live video applications. Similarly, with the 
trend towards distributed computing, video capture sys- 
tems which connect directly to a network are becoming 
desirable. A digital network camera would consist of 
a digital camera with added circuitry for network in- 
terfacing, distributed control and frame buffering. It 
is our hope that a subset of the Vidboard architecture 
and functionality could serve as a basis for the digital 
network camera of the future. Important architectural 
features are hardware support for pixel packing formats 
and dual frame buffering for rate adaptation. 


In addition to the basic tasks of video capture and 
transmission, the programmability of the Vidboard pro- 
vides support for other tasks related to digital video and 
distributed computing. The following tasks have been 
envisaged or are being implemented: 


e The extraction and decoding of closed-caption in- 
formation from a television signal. 

e A daughterboard for color dithering. 

e Video compression using the JPEG [14] standard. 

e Experimenting with the video header/descriptor 


formats under development by SMPTE [11]. 


Five Vidboards are currently in operation. Future 
plans call for a revision of the prototype board in 1993 
and the deployment of approximately twenty boards in 
a large-scale ViewStation system by 1994. 
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